Computer Adaptive Testing: Comparison of a Probabilistic Network Approach with Item Response Theory

Bayesian and probabilistic networks are claimed to offer powerful approaches to inferring an individual’s knowledge state from evidence of mastery of concepts or skills. A typical application where such tools can be useful is Computer Adaptive Testing (CAT). Bayesian networks have been proposed as an alternative to the traditional Item Response Theory (IRT), which has been the prevalent CAT approach for the last three decades. We review different Bayesian and probabilistic network approaches to modeling student ability assessment and compare the performance of one such approach, a probabilistic network approach named POKS, to the IRT two parameter logistic model. Experimental results over a 34 items UNIX test and a 160 items French language test show that both approaches can classify examinees as master or non master effectively and efficiently. Implications of these results for adaptive testing and student modeling are discussed.